Viewing-angle-dependent color/brightness correction for display system

ABSTRACT

A display system including a tracking subsystem configured to track one of a pose of a head of a viewer or a pose of an eye of the viewer. The display system further includes an image correction module configured to, for each pixel of at least a subset of pixels of an input image, determine a pixel view angle for the pixel based on the pose of the head or the pose of the eye, determine a corrected pixel value based on an input pixel value of the pixel in the input image and based on the pixel view angle; and provide the corrected pixel value for the pixel in an angle-corrected image corresponding to the input image. The display system further includes a display panel to display the angle-corrected image.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a U.S. National Stage under 35 U.S.C. § 371of International Patent Application Serial No. PCT/US2019/066028,entitled “VIEWING-ANGLE-DEPENDENT COLOR/BRIGHTNESS CORRECTION FORDISPLAY SYSTEM” and filed on Dec. 12, 2019, the entirety of which isincorporated by reference herein.

BACKGROUND

Many display devices have brightness or color responses that areview-angle-dependent due to radiometric and physical dependencies. Forexample, for liquid crystal displays (LCDs), bright pixels tend to dimas the view angle increases while dim pixels brighten as the view angleincreases due in large part to the angle-dependent phase retardation ofthe liquid crystal (LC) molecules. Similarly, many displays employpolarization files in order to reduce reflections. However, these filmshave some variation in extinction ratio based on wavelength and viewingangle. Brightness changes based on viewing angle also is often colordependent, and thus an increased view angle can lead to undesirablecolor shift desaturation and loss of contrast in the displayed imagery.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure may be better understood, and its numerousfeatures and advantages made apparent to those skilled in the art byreferencing the accompanying drawings. The use of the same referencesymbols in different drawings indicates similar or identical items.

FIG. 1 is a block diagram of a display system employing aview-angle-dependent color/brightness correction process in accordancewith some embodiments.

FIG. 2 is a diagram illustrating relationships between pixel view angleand luminance for a plurality of pixel values in accordance with someembodiments.

FIG. 3 is a block diagram of a hardware configuration of the displaysystem of FIG. 1 in accordance with some embodiments.

FIG. 4 is a flow diagram illustrating a method for view-angle-dependentimage correction in accordance with some embodiments.

FIG. 5 is a diagram illustrating a technique for mitigating quantizationeffects during view-angle-dependent image correction in accordance withsome embodiments.

FIG. 6 is a diagram illustrating a technique for mitigating clippingeffects during view-angle-dependent image correction in accordance withsome embodiments.

FIG. 7 is a diagram illustrating a display system employingview-angle-dependent image correction for multiple concurrent viewers inaccordance with some embodiments.

FIG. 8 is a diagram illustrating a display system employingview-angle-dependent correction for one or more viewers of a lightfielddisplay in accordance with some embodiments.

DETAILED DESCRIPTION

The physical characteristics of the pixel matrix or the overlyingpolarizer film of many types of display panels result in the color andbrightness of each pixel as perceived by a viewer as being dependent onthe view angle of the pixel relative to the viewer. As a result, thefurther a viewer is positioned from the central viewing axis of thedisplay panel (or the further to the edge a viewer's gaze directionextends in a near-eye display context), the more brightness and contrastare shifted or reduced. Disclosed herein are various embodiments ofdisplay systems that mitigate the spectral/angular response of suchdisplay panels through view-angle-dependent correction of the pixelvalues of an image before it is displayed at the display panel. In atleast one embodiment, a display system tracks a viewer's head (or aviewer's eye (e.g., as a gaze direction or other eye pose) in a near-eyedisplay implementation) and from this determines a current pose (of thehead and/or eyes) of the viewer relative to a display panel. For eachpixel of an input image intended for display at the display panel, thedisplay system determines a pixel view angle for the pixel from thecurrent pose. As used herein, the term “pose” can refer to one or bothof a position and an orientation. Thus, for example, the pose of an eyecan indicate one or both of the position of the eye or a gaze directionof the eye. The display system applies a correction to the pixel value(e.g., the red-blue-green (RGB) value) of the pixel based on its pixelview angle to generate an angle-corrected pixel value used in place ofthe original input pixel value. The correction applied to the pixelvalue represents a corrective adjustment to counteract aviewing-angle-dependent brightness or color response of thecorresponding pixel of the display panel. This process is repeated forsome or all of the pixels in the input image, and the resultingcorrected image is provided to the display panel for display. In thismanner, the brightness and contrast of each pixel in the image beingdisplayed is adjusted as a function of the viewer's view angle of thecorresponding pixel in the display panel.

For ease of illustration, example implementations are described withrespect to one display panel. This one display panel can represent, forexample, a single-panel display systems, such as a television,single-monitor computer system, smart phone, tablet computer, and thelike. However, in other embodiments, the display system can includemultiple display panels, in which case the view-angle-dependent imagecorrection processes described herein can be performed concurrently foreach display panel. To illustrate, in some embodiments, the displaysystem is a stereoscopic display system having one display panel foreach of the viewer's left eye and right eye, and in which the imagesprovided to the left-eye display panel are subjected to theview-angle-dependent image process described herein based on the gazedirection or pose of the left eye, while a similar process is performedfor the images provided to the right-eye display panel based on the gazedirection or pose of the right eye. As another example, the displaysystem could include a stereoscopic lightfield display system in which alenticular lens or other mechanism results in the provision of aseparate image for each of a viewer's left and right eyes (or in somecases, a separate image for each eye of each of a plurality of viewers),and the view-angle-dependent image correction process could be appliedfor each image for each eye based on a gaze direction or other poseindication of the corresponding eye of the viewer.

FIG. 1 illustrates a display system 100 employing a view-angle-dependentimage correction process in accordance with at least one embodiment. Thedisplay system 100 comprises any of a variety of electronic systemsutilized to display video or other imagery, including smartphones,tablet computers, laptop computers, computer monitors, video gamingdevices, televisions and other panel displays, as well as near-eyedisplay devices, such as augmented reality (AR) or virtual reality (VR)head-mounted devices, and the like. The display system 100 includes adisplay device 102 and a tracking subsystem 104. The display device 102includes at least one display panel 106 composed of a matrix of pixelsfor displaying sequences of images to at least one viewer. In theillustrated embodiment, the display device 102 is a single-displaysystem such as a television, computer monitor, smartphone, tabletcomputer, and the like. However, in other embodiments, the displaydevice 102 can include, for example, a virtual-reality (VR) headset, anaugmented-reality (AR) headset, or other near-eye stereoscopic displaysystem, and thus include, for example, two display panels 106, one foreach eye of the viewer, or a single display panel 106 that is used topresent separate images to each eye of the viewer, such as through alightfield display using a lenticular lens, through the use ofpolarizers or shutters, and the like.

The tracking subsystem 104 includes any of a variety of systems employedfor viewer head tracking (in the case of a non-near-eye implementation,e.g. pose of head tracking) or for eye gaze tracking (in the case of anear-eye implementation, e.g. gaze direction tracking of a viewer) asknown in the art. For example, the tracking subsystem 104 can include astereoscopic camera subsystem that utilizes reflected infrared (IR)light to detect the presence of a viewer's face and further to detectthe position and orientation (that is, the “pose”) of the viewer's faceto the display panel 106. For ease of reference, the systems andtechniques of the present disclosure generally are described in thecontext of head tracking, but these descriptions apply equally to gazetracking implementations for near-eye display configurations unlessotherwise noted. Further, for purposes of the following, the pose of theviewer's face (or more generally, the viewer's head) is described hereinwith reference to a central viewing axis 108 that is normal (that is,perpendicular) to an X-Y plane defined by the display panel 106 (thatis, extends along the Z-axis) at a point at the center of the displaypanel 106. However, in other embodiments, the pose may be definedrelative to a different fixed reference, such as a center point betweentwo cameras of the tracking subsystem 104, a specified corner of thedisplay panel 106, and the like.

As illustrated by diagram 110 illustrating a general overview of theoperation of the display system 100, at iterations of block 112 thetracking subsystem 104 continuously monitors for the presence of aviewer, and when detected, determines and periodically updates a currentpose of the viewer's head relative to the central view axis 108 (or gazedirection in near-eye implementations). Concurrently, at an iteration ofblock 114 the display device 102 obtains an input image from a sequenceof input images representative of a video stream. The input imagecomprises a plurality of pixel values, each pixel value corresponding toa pixel of the display panel 106. The input image can be a renderedimage, an image captured from a live event, or a combination thereof.The video stream can be generated at the display device 102 itself(e.g., as the video stream generated by execution of a video game at thedisplay device 102, as the video stream generated from the playback of avideo file stored in a local storage drive or a digital versatile disc(DVD), and the like). In other embodiments, the video stream isgenerated remotely and transmitted to the display device 102, such asvia a remote server and the like.

At a corresponding iteration of block 116, the display device 102 usesthe current pose of the viewer's head (or gaze direction) to determine,for each pixel of the display panel 106 (or each pixel of an identifiedsubset thereof) a corresponding pixel view angle for that pixel. Forexample, when the viewer's head is aligned with the central viewing axis108 at, for example, head pose 117-1, the viewing angle is determined tobe θ₁=0 degrees (deg.), whereas a viewer's head at head pose 117-2 has aviewing angle θ₂ represented by an angular component θ_(2X) along theX-axis and an angular component θ_(2Y) along the Y-axis (that is,θ₂=[θ_(2X), θ_(2Y)]). Likewise, a viewer's head at head pose 117-3 has aviewing angle θ₃ represented by an angular component θ_(3X) along theX-axis and an angular component θ_(3Y) along the Y-axis (that is,θ₃=[θ_(3X), θ_(3Y)]). Thus, the pixel view angle is a function of theviewing angle and distance of the viewer's head from the display panel106, as well as the (X,Y) position of the pixel within the display panel106.

The pixel view angle for each pixel and the pixel value of that pixel isthen used to determine a corrected pixel value that represents anadjustment of the pixel value so as to compensate for the expectedangle-dependent display behavior of the corresponding pixel when viewedat by the viewer at the corresponding viewing angle. That is,PV_(corr)=f(PV_(input), θ_(px), θ_(py)), where PV_(input) is the inputpixel value from the input image, θ_(px) and θ_(py) are the X-axiscomponent and Y-axis component, respectively of the pixel view angle ofthe pixel, and PV_(corr) is the corrected pixel value for the pixel. Asdescribed in greater detail herein, the correction to be applied given aparticular pixel value and pixel view angle can be determinedempirically, via simulation or modeling, and the like, and can beimplemented in any of a variety of transform structures, such as alook-up table (LUT), analytical function, and the like. As the impact oncolor and brightness of a displayed pixel is view-angle-dependent, theamount of correction applies typically increases as the pixel view angleincreases, for a given pixel value. To illustrate, when the viewer'shead is at pose 117-2, an input pixel for a pixel 118-1 of the displaypanel 106 that has a low or zero pixel view angle 119-1 typically wouldhave a lower amount of correction applied compared to the same inputpixel value for a pixel 118-2 of the display panel that has a relativelyacute pixel view angle 119-2.

The process of view-angle-based pixel value correction is repeated forsome or all of the pixels in the input image, resulting in anangle-corrected display image that is then output for display at thedisplay panel 106 at a corresponding iteration of block 120. With theper-pixel angular correction applied, the displayed image exhibitsimproved color saturation, brightness, and contrast as observed by theviewer at larger viewing angles compared to a similar display of theoriginal input image, particularly when the viewer's head is in aposition that results in a relatively oblique viewing angle.

FIG. 2 illustrates a chart 200 depicting example relationships betweenluminance and pixel view angle for various pixel values. In the chart200, the ordinate represents luminance of a pixel as observed by aviewer, the abscissa represents the pixel view angle of the pixelrelative to the viewer, and each of the depicted ten line plotsrepresents the relationship between luminance observed by the pixel fora corresponding pixel view angle for a corresponding pixel value of aset of ten pixel values. As illustrated by chart 200, the angularresponses differ for different pixel values. For example, line plot202-1 demonstrates that for a first pixel value, the pixel illuminatedin accordance with that first pixel value actually has a gain inperceived luminance as the pixel view angle increases, whereas line plot202-2 demonstrates an opposite relationship for a second pixelvalue—that is, that when the pixel is illuminated in accordance with thesecond pixel value, the observed luminance decreases as the pixel viewangle increases. Accordingly, in at least one embodiment, theangle-dependent corrections applied by the display system 100 areconfigured so as to attempt to maintain approximately the same defaultluminance (that is, the luminance exhibited at a pixel view angle of 0degrees) even as the pixel view angle increases. As demonstrated bybracket 204, this can involve a correction that increases the brightnessof the pixel value for those pixel values that experience luminance lossas pixel view angle increases, and as demonstrated by bracket 206, thisalso can involve a correction that decreases the brightness of the pixelvalue for those pixel values that experience luminance gain as pixelview angle increases.

FIG. 3 illustrates a simplified hardware configuration 300 of thedisplay system 100 of FIG. 1 in accordance with some embodiments. In thedepicted example, the display device 102 includes one or more centralprocessing units (CPUs) 302, one or more graphics processing units(GPUs) 304, at least one system memory 306, a frame buffer 308, and anangle-dependent image correction module 310 (referred to herein as“image correction module 310” for purposes of brevity). The displaypanel 106 includes a display controller 312 and a pixel matrix 314composed of a two-dimensional array of pixels, such as liquid crystal(LC)-based pixels, light emitting diode (LED)-based pixels, organic LED(OLED)-based pixels, or combinations thereof.

The illustrated configuration represents embodiments in which thedisplay device 102 is the source of the input video stream beingdisplayed. Accordingly, the CPU 302 operates to execute a video sourceapplication 316 stored in the system memory 306 that serves to directthe GPU 304 to render a stream of input images (e.g., input image 318),each of which is temporarily stored in the frame buffer 308. Forexample, the video source application 316 can include a video gameapplication that results in the rendering of a stream of video imagesrepresentative of the player's viewpoint into a video game scene. Asanother example, the video source application 316 can include a videodecoding application executed to decode encoded video data stored on ahard disc or other mass storage device (not shown) to generate asequence of decoded input images. In other embodiments, the video datais sourced by a remote device, such as a remote video server, in whichcase the video data is received at the display device 102 via a wirelessor wireless network, temporarily buffered, and decoded or otherwiseprocessed to generate the video stream composed of input images.

The image correction module 310 is implemented as hardcoded orprogrammable (e.g. free programmable, parametric programmable) logic, asone or both of the CPU 302 or GPU 304 executing correction software 324representing the functionality described herein, or a combinationthereof. In instances wherein the input images are rendered at thedisplay device 102, the correction software 324 can be implemented aspart of the rendering software. In instances where the input images aredecoded or otherwise recovered from previously-encoded video data, thecorrection software 324 can be implemented as part of the decodingsoftware The image correction module 310 operates to receive currentpose data 320 representative of the current pose of the viewer's head(or current gaze direction), and from this information, modify the pixelvalues of the input image 318 based on the viewing angle represented bythe viewer's current pose (or current gaze direction) to generate acorrected image 322. In embodiments wherein the input image 318 is beingrendered by the GPU 304, this correction process can be part of therendering process itself (e.g., as part of, or following, the pixelshader stage) such that the pixel values are corrected as the imageitself is being generated, and thus the input image 318 and thecorrected image 322 are the same image. In other embodiments, such aswhen the input image 318 is generated elsewhere (such as when thedisplay device 102 decodes encoded video data to obtain the input image318), the correction process can be applied as each pixel value of theinput image 318 is obtained or after the input image 318 is fullyobtained.

In at least one embodiment, the image correction module 310 determinesthe corrected pixel value for an input pixel value of the input image318 using a correction transform component 326 that represents aspecified relationship between input pixel value, pixel view angle, anddesired corrected pixel value (typically on a per-color-channel basis,as described in greater detail below). As such, the correction transformcomponent 326 can be implemented as, for example, a LUT that takes asinput the input pixel value and the pixel view angle and outputs acorrected pixel value, or as a software-implemented orhardware-implemented analytic function that receives these same inputsand provides a corresponding corrected pixel value. The operation of theimage correction module 310 and the correction transform component 326are described in greater detail below with reference to FIGS. 4-8 .

When angle-dependent correction has been applied to all applicable pixelvalues, the resulting corrected image 322 is transmitted to the displaycontroller 312, which then operates to drive the individual pixels ofthe pixel matrix 314 according to the corresponding corrected pixelvalue in the corrected image 322, and thus displaying an image to theviewer that has been corrected or otherwise adjusted to accommodate theviewer's particular viewing angle.

FIG. 4 illustrates an example method 400 for generation and display ofviewing-angle-corrected images at the display device 102 in accordancewith some embodiments. For ease of illustration, the method 400 isdescribed with reference to the hardware configuration 300 for thedisplay system 100 of FIG. 3 . The illustrated method 400 is composed oftwo subprocesses: an angle-dependent correction configuration subprocess402 and an angle-dependent correction application process 404. Theangle-dependent correction configuration subprocess 402 represents theprocess employed to initialize or otherwise configure theangle-dependent correction scheme to be applied by the display device102. Generally, the angle-dependent behavior exhibited by the displaypanel 106 depends on the technology or physical parameters of its pixelmatrix 314 (e.g., LED, LCD, OLED) or overlying polarizing film, themanufacturing process for employing that technology, as well asvariations in the implementation of the manufacturing process for thatparticular instance of the display panel 106.

As such, the luminance/pixel view angle relationship for a given pixelvalue typically is dependent on one or more of the display technology,manufacturing process, panel design, emitter natural angularillumination profile, etc. Accordingly, at block 406 a manufacturer,supplier, implementer, or user of the display device 102 determines theluminance-to-pixel-view-angle relationship for each applicable pixelvalue in the corresponding color space of the display device 102. In atleast one embodiment, this is determined on a per-color-channel basis.To illustrate, assuming the color space is an RGB color space, theluminance/angle relationship is separately determined for eachapplicable red color component value, each applicable green colorcomponent value, and each applicable blue color component value. In oneembodiment, the luminance/angle relationships are determined empiricallyvia testing/measurement of the display panel 106 itself, or arepresentative instance of the display panel 106. To illustrate, a blockof pixels of the display panel 106 can be configured to display acertain pixel value, and then a luminance/color meter can be positionedat different pixel view angles relative to the pixel block and aluminance/color measurement taken at each position, and from thesereadings a relationship between luminance and view angle, such as thoserepresented in the line plots of chart 200, can be determined for thecorresponding pixel value (or color component value thereof). In otherembodiments, the luminance/angle relationships are determined throughsimulation, modeling, or other such analysis.

With the luminance/angle relationships determined, at block 408 thecorrection transform component 326 is configured to reflect theserelationships. As described above, in at least one embodiment thecorrection transform component 326 is implemented as a LUT (e.g., LUT409) that takes as an inputs a pixel value (e.g., a color componentvalue) and a pixel view angle and provides a corrected pixel value inresponse, where the corrected pixel value is determined or selected suchthat when a pixel of the pixel matrix 314 is driven by the displaycontroller 312 based on the corrected pixel value, the light emitted bythe pixel and observed at the pixel view angle is substantially similarin brightness and color to the light emitted by the pixel when driven bythe input pixel value and observed head-on (that is, at a 0 degree viewangle). That is, the corrected pixel values of the LUT for a given inputpixel value are configured for each applicable pixel view angle so as toprovide an approximately constant luminance that is substantiallyindependent of pixel view angle. In other embodiments, the correctiontransform component 326 is implemented using a different structure, suchas an analytic function (e.g., analytic function 411), implemented insoftware or hardware, and configured in a similar manner so as todetermine a corresponding corrected pixel value for each input pixelvalue/pixel view angle tuple so that the resulting corrected pixelvalue, when used to drive a pixel of the pixel matrix 314, results asubstantially constant brightness or illumination for the correspondingpixel view angle. In at least one embodiment, this process is employedon a color basis by applying the above-described approach to each colorchannel independently; thus, color is preserved when the luminance ratioof the color elements (for example, the RGB channels) is preserved as afunction of viewing angle. If one of the corrected pixel values exceedsthe maximum value of the display (“clipping”), the color can bepreserved by reducing the non-clipped channels in accordance with thedisplay's gamma transformation such that the luminance ratio ispreserved, at the expense of local texture.

With the display device 102 so initialized, the method 400 moves to theangle-dependent correction application process 404 for providingangle-dependent correction to input images of a video stream prior todisplay. Accordingly, at block 410, the display device 102 obtains aninput image (e.g., input image 318) to be processed for a video streambeing displayed at the display device 102. As noted above, in someembodiments this input image is obtained through local rendering of theinput image at the display device 102, while in other embodiments theinput image is obtained through decoding or other such processing ofvideo data representative of the video stream, such as video datareceived from a remote server or other remote source via a networkconnection or video data obtained from a local mass storage device. Atblock 412, the image correction module 310 selects a pixel and obtainsthe pixel value of the input image for that pixel. In instances whereinthe input image is locally rendered, the pixel selected can be thecurrent pixel being rendered by the GPU 304. In instances wherein theinput image is generated remotely, the image correction module 310 canselect pixels from the input image in a predefined order, such asrow-by-row, column-by-column, block-by-block, or as each pixel value isdetermined during a decoding process or other image restoration process.As illustrated by block 414, the view tracking subsystem 104 monitorsthe pose of the viewer's head (or gaze direction/eye pose for near-eyedisplay implementations) and provides the head pose data 320 (FIG. 3 )representative of the currently-observed head pose (or gazedirection/eye pose) to the image correction module 310. Accordingly, atblock 416 the image correction module 310 determines the pixel viewangle for the pixel selected at block 412 based on the current viewingangle (that is, the angle of the viewer's head at the current poserelative to the central viewing axis 108, FIG. 1 , or other fixedreference) and the position of the pixel in the display panel 106relative to the central viewing axis 108 or other reference point. Asdescribed above, in one embodiment, the pixel view angle can berepresented using an X-axis component θ_(px) and a Y-axis componentθ_(py) relative to the X-Y plane formed by the display face of thedisplay panel 106. In other embodiments, it may be assumed that theviewer's head is relatively fixed along a particular axis, and thus thepixel view angle is instead represented by only a single angularcomponent relative to the other axis. To illustrate, in some instances,such as for fixed desktop-mounted televisions or computer monitors, thatthe viewer's head will remain at the same height relative to the displaypanel 106, but may roam horizontally as the viewer shifts position, andthus the Y-axis component of the pixel view angle may be fixed to 0degrees or otherwise disregarded.

With the pixel value and pixel view angle determined for the selectedpixel, at block 418 the image correction module 310 determines anangle-corrected pixel value based on these parameters. As explainedabove, this determination process is performed through the configurationof the correction transform component 326 to adjust for measured ortheorized angle-dependent luminance changes exhibited by a specified setof pixel values. To illustrate, for a LUT-based oranalytical-function-based implementation, the image correction module310 provides the pixel value and the pixel-view-angle as inputs andreceives as an output an angle-corrected pixel value. In someembodiments, this correction process is performed separately for eachcolor component of the pixel value. For example using the RGB colorspace, the range of red color component values is tested or simulated togenerate a corresponding set of LUT entries or analytic functionconfigurations representing the angle-corrected red color value to beused in place of each input red color value, and this same process canbe repeated for each of the green and blue color value ranges.

In other embodiments, the entire color component tuple representing acomplete pixel value is used to determine a corresponding correctedcolor component tuple for the corrected pixel value. To illustrateusing, for example, the 24-bit RGB color space, the RGB888 value RGB[r1,g1, b1] for the pixel can be converted to a corresponding YUV444 valueYUV[y1, u1, v1] in the YUV space using the standard RGB888-to-YUV444conversion, and then the original luminance component value y1 can bereplaced with a corrected luminance component value y2 based on theluminance-to-angle relationship (determined at block 406 and implementedat block 408) for the RGB888 value to generate a corrected YUV444 valueYUV[y2, u1, u1], which then can be converted back to a correspondingRGB888 value RGB[r2, g2, b2], which serves as the corrected pixel valuefor the input pixel value and pixel view angle combination.

At block 420, the image correction module 310 replaces or otherwisesubstitutes the corrected pixel value for the corresponding input pixelvalue in the corresponding angle-corrected image 322 (FIG. 3 ). Theprocess of blocks 412, 416, 418, and 420 is repeated for each pixel (ordesignated subset of pixels) of the input image 318. Once all applicablepixels have been corrected for viewing angle, the resultingangle-corrected image 322 is provided for display as described above,and method 400 returns to block 410 for processing the next input imagein the video sequence.

The process of adjusting pixel values based on corresponding pixel viewangles to pre-compensate for the angle-dependent color/brightness shiftsintroduced by viewing the pixels at their corresponding pixel viewangles can, without precaution, introduce undesirable visual artifacts,including banding due to quantization, clipping, and color unbalancing.FIGS. 5 and 6 illustrate various techniques for mitigating such issues.

As is well-known and reflected in Weber's Law of Just NoticeableDifference, changes in luminance are more readily detectable by a viewerfor dark, low-spatial-frequency textures than for lighter,higher-spatial-frequency textures. Accordingly, the quantization effectsof updating pixel values based on pixel-angle-view correction can leadto banding artifacts, particularly at lower luminance/color levels. FIG.5 illustrates a diagram 500 depicting example configurations of theimage correction module 310 to mitigate these quantization effects inaccordance with some embodiments. In one embodiment, the imagecorrection module 310 employs one or both of spatial dithering andtemporal dithering to reduce the quantization effects of angle-dependentpixel adjustments. This generally includes the application of aspatial/temporal dither pattern 502 to pixels so as to introduce “whitenoise” into the output and thus temper gradient effects. Any of avariety of spatial and/or temporal dither patterns can be employed, suchas a Poisson-Disc dither pattern, as may any of a variety of well-knowndithering algorithms. In such cases, the image correction module 310applies a specified dither pattern to the corrected pixel values as theyare output from the correction transform component 326 (FIG. 3 ), andthus introducing randomizing noise into the output pixel stream thatthen forms the corrected image 322. As banding effects are more likelyto be detected at darker pixel values, in some embodiments the imagecorrection module 310 can have a specified pixel value threshold suchthat the dithering pattern is applied only to those output correctedpixel values that fall below the threshold.

In other embodiments, the image correction module 310 can employ anintensity-weighted activation function 504 to activate angle-dependentcorrection as the pixel values exceed a specified threshold. Examples ofsuch activation functions include a hyperbolic tangent (tan h)activation function, a sigmoid activation function, a linear activationfunction, or any of a variety of asymptotic activation functions. Insuch instances, the image correction module 310 calculates the correctedpixel value as:PV_(corr_final)=PV_(input)AW*(PV_(corr_initial)−PV_(input))

where PV_(input) represents the input pixel value for a selected pixelof the input image 318, PV_(corr_initial) represents the angle-correctedpixel value obtained from the correction transform component 326 for theinput pixel value, AW represents an activation weight obtained from theactivation function 504 based on the input pixel value, andPV_(corr_final) represents the final angle-corrected pixel value that isused for the selected pixel in the angle-corrected image 322.

Another problem that may arise through the angle-dependent correctionprocess is the clipping of pixels when the output intensity level for anangle-corrected pixel value exceeds the maximum level supported by thedisplay panel 106. FIG. 6 illustrates a diagram 600 of various solutionsthat can be employed by the image correction module 310 to address suchclipping issues. As a general overview, these solutions involve thepre-darkening of pixel values so as to reduce or eliminate thepossibility of a corrected pixel value exceeding the maximum supportedintensity level. In one embodiment, the pixel values are pre-darkenedstatically according to the corresponding pixel positions. Toillustrate, the image correction transform can utilize a disk-shapeddarkening pattern 602 having a plurality of concentric contours 604logically overlying the display area 606 of the pixel matrix 314, witheach contour 604 being associated with a different level ofpre-darkening and with the amount of pre-darkening increasing from theexterior-most contour 604 to the inner-most contour 604. Thus, the levelof pre-darkening applied to the pixel value of a given pixel depends onthe particular contour 604 in which the pixel is located within thedarkening pattern 602. The pre-darkening pattern can take other shapes,such as columns of contours, rows of contours, a 2D grid of contours,and the like.

The pre-darkening can be a fixed value for a given pre-darkening level.For example, for RGB888 implementation in which color component valuesrange from 0 to 255, the pre-darkening applied for a pixel in theinner-most contour 604 may be reducing each color component by 20,whereas the pre-darkening applied for a pixel in the outer-most contourmay be reducing each color component by only 5. In other embodiments,the pre-darkening is employed as a scaling value, such as a scalingvalue between, for example, 0.8 and 1. In some embodiments, thepre-darkening process is applied to the input pixel values, such thatthe input pixel value is pre-darkened and the resulting pre-darkenedinput value is input to the correction transform component 326 to obtaina corresponding angle-corrected pixel value. In other embodiments, thepre-darkening process is applied to the angle-corrected pixel values asthey are output from the correction transform component 326. In eitherinstance, the pre-darkening pattern 602 can be implemented in a LUT orsimilar data structure in which the pixel position is provided as aninput, and the corresponding pre-darkening value to the applied to thepixel value is provided as an output.

Rather than using static position-based pre-darkening, in otherembodiments the image correction module 310 uses a uniform pre-darkeningthat is based on the viewer's viewing angle, rather than pixel position.To illustrate, the image correction module 310 can use a specifiedviewing angle/pre-darkening relationship, which is illustrated in FIG. 6as a viewing angle/pre-darkening chart 608 with the viewing anglesrepresented in the abscissa and corresponding pre-darkening scalefactors represented in the ordinate, and which can be implemented as aLUT or other similar structure. In this approach, each pixel value isscaled down by the same scale factor, which is determined based on theviewing angle, and which can range between a minimum and maximum scalingfactor (e.g., 0.8 to 1.0), with increased pre-darkening scaling asviewing angle increases. As with the position-based static pre-darkeningprocess described above, this pre-darkening scaling can be applied tothe input pixel so that a pre-darkened input pixel is used as a basisfor determining an angle-corrected pixel value, or the pre-darkeningscaling can be applied to the angle-corrected pixel value determined forthe original pixel value.

Some display devices provide segmented backlighting in which differentregions, such as columns, rows, or 2D blocks, of the backlight can beindependently controlled and thus the amount of backlighting can varybetween the different regions. In instances wherein the display device102 supports such segmented backlighting, the display device 102 canleverage this segmented backlighting capability to, in effect, recapturesome of the intensity lost through the pre-darkening process so as toprovide a nearly “flat” illumination after pre-darkening. To illustrate,in a configuration whereby the pre-darkening pattern is a column-basedcontour pattern and the backlight 610 of the display device 102 likewiseis segmented into columns, the backlighting intensity of each segmentcan be increased proportional to the amount of pre-darkening employedfor the contour co-positioned with the backlight segment. As anotherexample, when using the angle-dependent static pre-darkening, thebacklighting intensity can be varied based on the angle of eachbacklight segment relative to the viewer's head pose such that the lessacute the angle (that is, the closer to the viewer's head), the lowerbacklighting intensity is applied. To illustrate, when the viewer's headis centered in the display, the backlight segments closer to the centerare controlled to provide less backlighting, while those further fromthe center are controlled to provide more backlighting, as illustratedin FIG. 6 . When the viewer's head is to one side of the display, thebacklight segments on that side are controlled to provide lessbacklighting, while those on the other side are controlled to providemore backlighting. By varying the backlight intensity in this manner andeffectively inversely to the pre-darkening applied to the pixels in thecorresponding region, the display device 102 can maintain asubstantially flat illumination while mitigating the chances ofclipping.

Clipping is particularly problematic when one color component of a pixelvalue clips while the other color components remain unclipped, and thusresults in a changed luminance ratio between the color component values.In such instances, the image correction module 310 can reweight thecolor components of the corrected pixel value so as to maintain the sameluminance ratio of the original pixel value. In doing so, variations intexture may be lost, but the color of the pixel itself will not changefrom input to corrected value.

FIG. 7 illustrates an implementation of the display system 100 forproviding angle-dependent image correction when there are multiplesimultaneous viewers. In this approach, the tracking subsystem 104tracks the head poses of the multiple viewers (e.g., viewers 702, 704)and the image correction module 310 determines a combined head pose 706which is then used by the image correction module 310 to applyangle-dependent image correction to an input image 318 so as to generatean angle-corrected image 322 for display at the display device 102. Thecombined head pose 706 can be determined from the individual head posesin any of a variety of ways. To illustrate, the individual head posescan be simply averaged. In other embodiments, a weighted averagingalgorithm can be employed, whereby each head pose is weighted based onits viewing angle. Still further, certain thresholds can be employed soas to eliminate head poses at extreme viewing angles (so as to preventthe correspondingly extreme angle-dependent corrections from providing adetracting result for a more centrally positioned viewer), and so forth.

Lightfield displays are particularly suited to multiple-viewerexperiences in that they can provide a 3D view of a scene simultaneouslyto multiple viewers (either a single image to each viewer, or in someimplementations, a separate image to each eye of a viewer, and withdifferent left-eye/right-eye image pairs provided to different viewersbased on head pose and/or gaze direction of each viewer). As illustratedby FIG. 8 , the angle-dependent image correction process can be employedfor a lightfield display device 802 (one embodiment of the displaydevice 102, FIG. 1 ) in that the head pose for multiple simultaneousviewers (e.g., viewers 804, 806) via the tracking subsystem 104. Asource system generates or obtains an input lightfield 818 (oneembodiment of the input image 318, FIG. 3 ) and the display device 802performs a separate angle-dependent correction for each viewer (or foreach eye of each viewer if separate images are provided to each eye ofthe viewer) using the viewer's corresponding head pose (e.g., head poses808, 810 for viewers 804, 806, respectively) and/or eye pose/gazedirection on the lightfield pixel data representing the rays expected tobe perceived by the viewer in the viewer's corresponding position,resulting in a corrected lightfield 822 (one embodiment of the correctedimage 322, FIG. 3 ) having angle-dependent color corrections separatelyemployed for each tracked viewer, which is then provided for display atthe lightfield display device 802.

In some embodiments, certain aspects of the techniques described abovemay implemented by one or more processors of a processing systemexecuting software. The software comprises one or more sets ofexecutable instructions stored or otherwise tangibly embodied on anon-transitory computer readable storage medium. The software caninclude the instructions and certain data that, when executed by the oneor more processors, manipulate the one or more processors to perform oneor more aspects of the techniques described above. The non-transitorycomputer readable storage medium can include, for example, a magnetic oroptical disk storage device, solid state storage devices such as Flashmemory, a cache, random access memory (RAM) or other non-volatile memorydevice or devices, and the like. The executable instructions stored onthe non-transitory computer readable storage medium may be in sourcecode, assembly language code, object code, or other instruction formatthat is interpreted or otherwise executable by one or more processors.

A computer readable storage medium may include any storage medium, orcombination of storage media, accessible by a computer system during useto provide instructions and/or data to the computer system. Such storagemedia can include, but is not limited to, optical media (e.g., compactdisc (CD), digital versatile disc (DVD), Blu-Ray disc), magnetic media(e.g., floppy disc, magnetic tape, or magnetic hard drive), volatilememory (e.g., random access memory (RAM) or cache), non-volatile memory(e.g., read-only memory (ROM) or Flash memory), ormicroelectromechanical systems (MEMS)-based storage media. The computerreadable storage medium may be embedded in the computing system (e.g.,system RAM or ROM), fixedly attached to the computing system (e.g., amagnetic hard drive), removably attached to the computing system (e.g.,an optical disc or Universal Serial Bus (USB)-based Flash memory), orcoupled to the computer system via a wired or wireless network (e.g.,network accessible storage (NAS)).

Note that not all of the activities or elements described above in thegeneral description are required, that a portion of a specific activityor device may not be required, and that one or more further activitiesmay be performed, or elements included, in addition to those described.Still further, the order in which activities are listed are notnecessarily the order in which they are performed. Also, the conceptshave been described with reference to specific embodiments. However, oneof ordinary skill in the art appreciates that various modifications andchanges can be made without departing from the scope of the presentdisclosure as set forth in the claims below. Accordingly, thespecification and figures are to be regarded in an illustrative ratherthan a restrictive sense, and all such modifications are intended to beincluded within the scope of the present disclosure.

In the following some examples are described.

Example 1: A display system comprising:

-   -   a tracking subsystem configured to track one of a pose of a head        of a viewer or a pose of an eye of the viewer;    -   an image correction module configured to:        -   for each pixel of at least a subset of pixels of an input            image:            -   determine a pixel view angle for the pixel based on the                pose or the gaze direction;            -   determine a corrected pixel value based on an input                pixel value of the pixel in the input image and based on                the pixel view angle; and            -   provide the corrected pixel value for the pixel in an                angle-corrected image corresponding to the input image;                and    -   a display panel to display the angle-corrected image.

Example 2: The display system of example 1, wherein the corrected pixelvalue represents an corrective adjustment to counteract anviewing-angle-dependent brightness or color response of thecorresponding pixel of the display panel.

Example 3: The display system of example 1 or 2, wherein the correctedpixel value is at least one of luminance or color.

Example 4: The display system of at least one of the preceding examples,further comprising:

-   -   a correction transform component configured based on        relationships between pixel luminance and view angle of a pixel        of the display panel for a plurality of pixel values; and    -   wherein the image correction module determines the corrected        pixel value using the correction transform.

Example 5: The display system of example 4, wherein the relationshipsare determined based on one of empirical measurement, simulation, ormodeling of the display panel.

Example 6: The display system of example 4 or 5, wherein the correctiontransform component comprises one of a look up table or an analyticalfunction.

Example 7: The display system of at least one of the preceding examples,wherein the image correction module is configured to apply at least oneof a spatial dithering pattern or a temporal dithering pattern tocorrected pixel values of the angle-corrected image.

Example 8: The display system of at least one of the preceding examples,wherein the image correction module is configured to use an activationfunction to selectively correct the input pixel value.

Example 9: The display system of at least one of the preceding examples,wherein the image correction module is configured to pre-darken pixelvalues of one of the input image or the angle-corrected image using oneof: a uniform pre-darkening of the pixel values based on the pose or thegaze direction; and a static pre-darkening of the pixel values based onpositions of pixels within the input image.

Example 10: The display system of example 9, further comprising:

-   -   a backlight comprising a plurality of independently-controllable        segments, wherein the backlight is controlled to compensate for        the pre-darkening of the pixels values on a segment-by-segment        basis.

Example 11: The display system of at least one of the precedingexamples, wherein the display panel comprises a display panel of one of:a television; a smartphone; a tablet computer; a notebook computer; acomputer monitor, a near-eye display device; and a lightfield displaydevice.

Example 12: The display system of at least one of the precedingexamples, wherein the tracking system comprises a camera-basedhead-tracking subsystem.

Example 13: A method for implementation in a display system, the methodcomprising:

-   -   tracking one of a pose of a viewer's head or a pose of an eye of        a viewer's eyes;    -   for each pixel of at least a subset of pixels of an input image:        -   determining a pixel view angle for the pixel based on the            pose or the gaze direction;        -   determining a corrected pixel value based on an input pixel            value of the pixel in the input image and based on the pixel            view angle; and        -   providing the corrected pixel value for the pixel in an            angle-corrected image corresponding to the input image; and    -   providing the angle-corrected image for display at a display        panel of the display system.

Example 14: The method of example 13, wherein the corrected pixel valuerepresents an corrective adjustment to counteract anviewing-angle-dependent brightness or color response of thecorresponding pixel of the display panel.

Example 15: The display system of example 13 or 14, wherein thecorrected pixel value is at least one of luminance or color.

Example 16: The method of at least one of the examples 13 to 15, furthercomprising:

-   -   configuring a correction transform component based on        relationships between pixel luminance and view angle of a pixel        of the display panel for a plurality of pixel values; and    -   wherein determining the corrected pixel value comprises        determining the corrected pixel value using the correction        transform.

Example 17: The method of example 16, further comprising:

-   -   determining the relationships based on one of empirical        measurement, simulation, or modeling of the display panel.

Example 18: The method of at least one of the examples 13 to 17, furthercomprising:

-   -   applying at least one of a spatial dithering pattern or a        temporal dithering pattern to corrected pixel values of the        angle-corrected image.

Example 19: The method of at least one of the examples 13 to 18, furthercomprising:

-   -   selectively correcting the input pixel value based on an        activation function.

Example 20: The method of at least one of the examples 13 to 19, furthercomprising:

-   -   pre-darkening pixel values of one of the input image or the        angle-corrected image using one of: a uniform pre-darkening of        the pixel values based on the viewer's pose; and a static        pre-darkening of the pixel values based on positions of pixels        within the input image.

Example 21: The method of example 20, further comprising:

-   -   controlling individually-controllable segments of a backlight to        compensate for the pre-darkening of the pixels values on a        segment-by-segment basis.

Example 22: The method of at least one of the examples 13 to 21, whereinthe display panel comprises a display panel of one of: a television; asmartphone; a tablet computer; a notebook computer; a computer monitor,a near-eye display device; and a lightfield display device.

Benefits, other advantages, and solutions to problems have beendescribed above with regard to specific embodiments. However, thebenefits, advantages, solutions to problems, and any feature(s) that maycause any benefit, advantage, or solution to occur or become morepronounced are not to be construed as a critical, required, or essentialfeature of any or all the claims. Moreover, the particular embodimentsdisclosed above are illustrative only, as the disclosed subject mattermay be modified and practiced in different but equivalent mannersapparent to those skilled in the art having the benefit of the teachingsherein. No limitations are intended to the details of construction ordesign herein shown, other than as described in the claims below. It istherefore evident that the particular embodiments disclosed above may bealtered or modified and all such variations are considered within thescope of the disclosed subject matter. Accordingly, the protectionsought herein is as set forth in the claims below.

What is claimed is:
 1. A display system comprising: a tracking subsystemconfigured to track at least one of a pose of a head or a pose of an eyeof a viewer; an image correction module configured to: for each pixel ofat least a subset of pixels of an input image: determine a pixel viewangle for the pixel based on the pose of the head or the pose of theeye; determine a corrected pixel value based on an input pixel value ofthe pixel in the input image and based on the pixel view angle; andprovide the corrected pixel value for the pixel in an angle-correctedimage corresponding to the input image; and apply at least one of aspatial dithering pattern or a temporal dithering pattern to correctedpixel values of the angle-corrected image; and a display panel todisplay the angle-corrected image.
 2. The display system of claim 1,wherein the corrected pixel value represents a corrective adjustment tocounteract a viewing-angle-dependent brightness or color response of acorresponding pixel of the display panel.
 3. The display system of claim1, wherein the corrected pixel value is at least one of luminance orcolor.
 4. The display system of claim 1, further comprising: acorrection transform component configured based on relationships betweenpixel luminance and view angle of a pixel of the display panel for aplurality of pixel values; and wherein the image correction moduledetermines the corrected pixel value using the correction transform. 5.The display system of claim 4, wherein the relationships are determinedbased on one of empirical measurement, simulation, or modeling of thedisplay panel.
 6. The display system of claim 4, wherein the correctiontransform component comprises one of a look up table or an analyticalfunction.
 7. The display system of claim 1, wherein the image correctionmodule is configured to use an activation function to selectivelycorrect the input pixel value.
 8. The display system of claim 1, whereinthe image correction module is configured to pre-darken pixel values ofone of the input image or the angle-corrected image using one of: auniform pre-darkening of the pixel values based on the pose of the heador the pose of the eye; and a static pre-darkening of the pixel valuesbased on positions of pixels within the input image.
 9. The displaysystem of claim 8, further comprising: a backlight comprising aplurality of independently-controllable segments, wherein the backlightis controlled to compensate for the pre-darkening of the pixel values ona segment-by-segment basis.
 10. The display system of claim 1, whereinthe display panel comprises a display panel of one of: a television; asmartphone; a tablet computer; a notebook computer; a computer monitor,a near-eye display device; and a lightfield display device.
 11. Thedisplay system of claim 1, wherein the tracking subsystem comprises acamera-based head-tracking subsystem.
 12. A method for implementation ina display system, the method comprising: tracking at least one of a poseof a viewer's head or a pose of a viewer's eye; for each pixel of atleast a subset of pixels of an input image: determining a pixel viewangle for the pixel based on the pose of the viewer's head or the poseof the viewer's eye; determining a corrected pixel value based on aninput pixel value of the pixel in the input image and based on the pixelview angle; and providing the corrected pixel value for the pixel in anangle-corrected image corresponding to the input image; applying atleast one of a spatial dithering pattern or a temporal dithering patternto corrected pixel values of the angle-corrected image; and providingthe angle-corrected image for display at a display panel of the displaysystem.
 13. The method of claim 12, wherein the corrected pixel valuerepresents an corrective adjustment to counteract anviewing-angle-dependent brightness or color response of thecorresponding pixel of the display panel.
 14. The method of claim 12,wherein the corrected pixel value is at least one of a luminance valueor a color value.
 15. The method of at least one of claim 12, furthercomprising: configuring a correction transform component based onrelationships between pixel luminance and view angle of a pixel of thedisplay panel for a plurality of pixel values; and wherein determiningthe corrected pixel value comprises determining the corrected pixelvalue using the correction transform.
 16. The method of claim 15,further comprising: determining the relationships based on one ofempirical measurement, simulation, or modeling of the display panel. 17.The method of claim 12, further comprising: selectively correcting theinput pixel value based on an activation function.
 18. The method ofclaim 12, further comprising: pre-darkening pixel values of one of theinput image or the angle-corrected image using one of: a uniformpre-darkening of the pixel values based on the pose of the viewer's heador the pose of the viewer's eye; and a static pre-darkening of the pixelvalues based on positions of pixels within the input image.
 19. Themethod of claim 18, further comprising: controllingindividually-controllable segments of a backlight to compensate for thepre-darkening of the pixel values on a segment-by-segment basis.
 20. Themethod of claim 12, wherein the display panel comprises a display panelof one of: a television; a smartphone; a tablet computer; a notebookcomputer; a computer monitor, a near-eye display device; and alightfield display device.